Skip to content

refactor(profiler): add RAII step scope for service-wide timeline#1133

Merged
LLLLKKKK merged 1 commit into
mainfrom
feat/profiler-step-scope-v2
Jun 30, 2026
Merged

refactor(profiler): add RAII step scope for service-wide timeline#1133
LLLLKKKK merged 1 commit into
mainfrom
feat/profiler-step-scope-v2

Conversation

@psmarter

Copy link
Copy Markdown
Collaborator

Motivation

Service-wide timeline profiling should work consistently across engine types, including embedding workloads such as BERT. This PR also replaces the manual profiler tick() flow with an RAII step scope so the profiling step boundary is explicit and exception-safe.

Changes

  • Add StepWindowProfiler::StepScope to bracket one engine step.
    • Calls beginStep() on construction.
    • Calls endStep() on destruction.
    • Catches and logs endStep() exceptions without propagating them into the engine
      path.
  • Add StepWindowProfiler::configureFromConfig() for service-wide timeline setup from
    ProfilingDebugLoggingConfig.
  • Wire service-wide timeline profiling into:
    • NormalEngine
    • EmbeddingEngine
  • Add service-wide timeline config fields:
    • --gen_timeline_start_step / GEN_TIMELINE_START_STEP
    • --gen_timeline_num_steps / GEN_TIMELINE_NUM_STEPS
    • --gen_timeline_trace_name / GEN_TIMELINE_TRACE_NAME
  • Keep pickle backward compatibility for ProfilingDebugLoggingConfig.
    • Old 12-field tuple is still accepted.
    • New 15-field tuple includes the timeline window fields.
  • Update profiling docs to describe service-wide timeline as a step-window trace.

Validation

  • git diff --check origin/main..HEAD
  • python3 -m py_compile rtp_llm/server/server_args/profile_debug_logging_group_args.py
  • bazelisk build //rtp_llm/cpp/engine_base:profiler //rtp_llm/cpp/normal_engine:normal_engine//rtp_llm/cpp/embedding_engine:embedding_engine --config=rocm

Build result:

INFO: Build completed successfully

@psmarter psmarter requested a review from LLLLKKKK as a code owner June 23, 2026 09:40
Copilot AI review requested due to automatic review settings June 23, 2026 09:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@LLLLKKKK

Copy link
Copy Markdown
Collaborator

AI Code Review - PR #1133

Status: LGTM

Summary: P0/0 · P1/0 · P2/1 · P3/0

lgtm ready to ci

Non-blocking Suggestions

P2

  • 新增 StepScope RAII 模式和配置字段缺少测试覆盖 @ rtp_llm/cpp/engine_base/TorchProfiler.h:81
    • 建议:建议至少添加:(1) StepScope RAII 构造/析构正确调用 beginStep/endStep 的验证;(2) step 计数达到 num_steps 后 enabled_ 自动清除的测试;(3) configureFromConfig 在 gen_timeline_sync=false 时 no-op 的单元测试;(4) endStep 中 disable 路径调用 stopProfiler;(5) reconfigure 重置 waited_steps/profiled_steps;(6) pickle round-trip 测试覆盖 size=12 和 size=15 两种格式。可用 mock TorchProfile 避免 Kineto 依赖。

Checklist Violations (1 fail / 72 total)

General Principles Checklist

  • [6.1] Tests — 新逻辑有聚焦单测 + 相关集成/smoke 测试 → issue 新增 StepScope RAII 模式和配置字段缺少测试覆盖
    StepScope RAII guard、configureFromConfig()、beginStep/endStep 状态机均无测试。grep 整个 test/ 目录无 StepWindowProfiler 引用。新增三个 config 字段的传播链也缺少覆盖。

Strengths

  • RAII StepScope 从根本上消除了手动 tick() 的不对称调用风险——旧代码 EmbeddingEngine 只调一次 tick()、NormalEngine 调两次,语义不一致;新 RAII 统一了两个引擎的 profiler 生命周期
  • StepScope 析构函数正确捕获 std::exception 和 catch-all 两级异常并记录 ERROR 日志,符合 C++ 析构不应抛异常的最佳实践
  • pickle 反序列化通过 tuple size 分支(12/15)实现向后兼容,旧格式使用 C++ struct 默认值,平滑升级
  • configureFromConfig() 统一了两个引擎的配置入口,消除了 gen_timeline_sync 检查的重复代码

Comment thread rtp_llm/cpp/pybind/ConfigInit.cc
@LLLLKKKK LLLLKKKK enabled auto-merge (rebase) June 26, 2026 08:04
@psmarter psmarter force-pushed the feat/profiler-step-scope-v2 branch from fff03e2 to 8dfd553 Compare June 26, 2026 17:12
@wht21

wht21 commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

internal source has been updated, please review the changes!

@LLLLKKKK

Copy link
Copy Markdown
Collaborator

AI Code Review - PR #1133

Status: LGTM

Summary: P0/0 · P1/0 · P2/3 · P3/2

lgtm ready to ci

Non-blocking Suggestions

P2

  • 缺少 StepWindowProfiler / StepScope 单元测试 @ rtp_llm/cpp/engine_base/TorchProfiler.cc:95
    • 建议:为 StepWindowProfiler 新增单元测试,覆盖: (1) StepScope RAII 正常 begin/end;(2) start_step>0 的 warmup 延迟;(3) num_steps 到达后自动停止;(4) configureFromConfig gen_timeline_sync=false 的 no-op 路径;(5) StepScope 析构异常安全。可用 mock TorchProfile 避免 Kineto 依赖。
  • endStep 中对 enabled_ 的双重 relaxed load 在 disable 边界可能多执行一次 stopProfiler @ rtp_llm/cpp/engine_base/TorchProfiler.cc:194
    • 建议:将 enabled_.load(relaxed) 缓存到局部变量后复用,与 beginStep() 的模式保持一致。例如:const bool en = enabled_.load(relaxed); if (!en && !has_profiler_.load(relaxed)) return; if (!en) { stopProfiler("disabled"); return; }
  • beginStep 中双重获取 mutex 可合并为单次加锁 @ rtp_llm/cpp/engine_base/TorchProfiler.cc:152
    • 建议:将 reconfigure 处理和 profiler start 合并到同一个 lock_guard 作用域中,消除中间释放锁的窗口。

P3

  • EmbeddingEngine 在 profile_step scope 内调用 cudaSyncAndCheck 可能影响 profiling 精度 @ rtp_llm/cpp/embedding_engine/EmbeddingEngine.cc:131
    • 建议:确认这是有意设计(embedding 要记录 sync 开销)还是应与 NormalEngine 保持一致将 cudaSyncAndCheck 移到 scope 外。如有意保留,建议加注释说明。
  • 文档未提及新增的 --gen_timeline_start_step/num_steps/trace_name 参数 @ docs/references/profiling.md:25
    • 建议:在 Service-Wide Timeline 段落后补充参数说明表,列出三个新参数的名称、环境变量、默认值和用途。

Checklist Violations (3 fail / 72 total)

General Principles Checklist

  • [6.1] Software Engineering — DRY:重复非平凡逻辑被抽取或显式复用 → checklist-only
    beginStep 和 endStep 有相同的 fast-path 检查+stopProfiler 逻辑(两次 atomic load + stopProfiler 调用),属于轻度重复。因在 hot path 且代码量小,属于性能路径的合理重复,不升级为 issue。
  • [6.1] Tests — 新逻辑有聚焦单测 + 相关集成/smoke 测试 → issue 缺少 StepWindowProfiler / StepScope 单元测试
    新增 StepScope RAII、beginStep/endStep 拆分、configureFromConfig 等核心接口缺少单元测试。现有 smoke 测试间接覆盖端到端路径,但白盒行为(start_step 延迟、num_steps 停止、reconfigure 重启)无直接测试。
  • [6.1] Tests — 边界 case 覆盖(空、单元素、最大值) → issue 缺少 StepWindowProfiler / StepScope 单元测试
    start_step=0 立即开始、num_steps=1 单步停止、num_steps=0 边界、configure 在 profiler 已 active 时被忽略等边界场景无测试验证。

Strengths

  • RAII StepScope 设计保证了 Kineto 线程亲和性和异常安全,即使 process() 抛异常也能正确推进 profiler 状态机,消除了旧 tick() 两次调用的认知负担
  • pickle 向后兼容处理得当,接受 tuple size 12(旧版)和 15(新版),避免滚动升级时反序列化失败
  • configureFromConfig 统一了启动时 profiler 配置路径,消除了 NormalEngine/EmbeddingEngine 中的重复字段提取逻辑
  • StepScope 析构函数中的 try/catch 确保异常不传播到 engine loop 外层导致 std::terminate,符合 C++ 析构函数 best practice

@wht21

wht21 commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

internal source has been updated, please review the changes!

@LLLLKKKK LLLLKKKK merged commit a0ee9c6 into main Jun 30, 2026
7 of 9 checks passed
@siluzhou

Copy link
Copy Markdown
Collaborator

GEN_TIMELINE_START_STEP 环境变量是临时做法,线上部署设置环境变量才支持timeline是很重的改动,最好参考normal engine,做成请求级别的参数

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants